Analyzing the dynamic behavior and performance of a complex software system is difficult. Typically, analysis of a software system is achieved by gathering data at each system call and post-processing the data. The following is a brief description of conventional tracing frameworks used to analyze software.
The conventional tracing frameworks were typically composed of various independent software modules. The primary source of information accessed by the conventional tracing frameworks is the kernel. The conventional tracing frameworks typically include a trace facility, a trace module, a daemon, and an offline data analysis and processing component. The trace facility gathers information from various components of the kernel and forwards events to the trace module. The trace module subsequently logs the events in its buffer. Periodically, the trace daemon reads the events from the trace module buffer and commits the recorded events into a user-provided file.
When diagnosing system failure, one often wishes to know the events leading up to failure. Moreover, in cases where reproducing failure from the events leading up to the failure, can take hours or days, one may wish to only keep the most recent data. To allow for this, tracing frameworks have historically implemented ring buffers. A ring buffer is a buffer, which upon filling, proceeds to return to the top of the buffer and overwrites the oldest data as new data is received. To enable a ring buffering scheme, conventional tracing frameworks have typically required that all data stored in the buffer to be of uniform size.
To process data within a ring buffer correctly, i.e., extracting information in chronological order for oldest data to youngest data, one must know the position of the oldest record. Processing starts at the oldest record in the buffer and continues until the end of the buffer is reached. If the ring buffer has not wrapped (i.e., proceeded to overwrite old data), then the processing of the ring buffer is completed upon reaching the end of the buffer. However, if the ring buffer has wrapped, the processing resumes at the beginning of the buffer (i.e., at zero offset), and continues until the youngest record in the ring buffer is processed. Since the data was typically of a fixed-length, the position of the oldest record was either at offset zero if the ring buffer had not wrapped, or at the current offset if the ring buffer had wrapped.